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Abstract 

A writing-intensive, upper-level undergraduate course which integrates 
content, context, collaboration, and communication in a unique fashion, is 
described. The topic of the seminar is “Scientific Writing in Chemistry ” and 
an assignment-based curriculum was developed to instruct students on best 
practices in all aspects of science communication and to educate students 
about the scientific publication process and peer review. To effectively teach 
students how to understand science, both the content and the process must be 
included. Peer review is an integral and essential part of the process of 
science and the peer review tasks in the course described in this paper evolve 
from rubric-based peer assessments to free-format peer review. The 
curriculum was developed for a semester-long, three-hour seminar with 
limited enrolment. The curriculum was taught in the Spring semesters of 
2010 - 2014 and enrolment data and results of evaluations collected over 
four years are presented to demonstrate the success of the implementations. 
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Introduction 

Building on a century of general education policy (Boyer & Levine, 1981), the 
National Science Foundation (NSF, 1996) recommended that STEM teachers: 

... model good practices that increase learning; start with the student’s 
experience, but have high expectations within a supportive climate; and 
build inquiry, a sense of wonder and the excitement of discovery, plus 
communication and teamwork, critical thinking, and life-long learning 
skills into learning experiences, (p. iv) 

Within this learning framework, communication includes literacy (Robinson, 
McKenna, & Conradi, 2011), which in the 21st century increasingly means 
scientific literacy (Pearson, Moje, & Greenleaf, 2010). This requires an 
appreciation of the scientific process by the general public (Schwartz, 2007) as 
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well as public engagement and effective science communication by scientists 
(Nisbet & Scheufele, 2009). 

We developed Chemistry is in the News ( CIITN) to teach chemistry in the context 
of real-world issues and to expose students to all aspects of science 
communication. The CIITN activities consist of the study, creation and peer 
review of online projects based on news articles from the popular press. The 
activities aim to connect real-world social, economic and political issues to the 
learning of chemistry. The CIITN pedagogical framework (Carson, Hodgen, & 
Glaser, 2006; Glaser & Carson, 2005; Glaser & Poole, 1999) and technical issues 
(Wu & Glaser, 2004) of the implementation have been described and reviewed 
(Carson, Hume, Sui, Schelble, & Glaser, 2009; Glaser, 2003, 2013), and results of 
evaluations (Carson & Glaser, 2010; Hume, Carson, Hodgen, & Glaser, 2006) 
have been reported. The entire CIITN peer review is conducted by students and 
includes an assessment framework for both individual and group assessment, 
rubrics to guide peer review and an intragroup peer review tool. CIITN was 
developed for large, lower-division (first and second year) university courses. 
CIITN has also been implemented in high schools in the United States. 

Here, we report a more recent curriculum innovation which embraces the spirit 
and expands on the concepts of CIITN to educate upper-division (third and fourth- 
year) science majors about the scientific process, scientific writing, scientific peer 
review, and professional issues. The framework of an assignment-based 
curriculum of a writing-intensive, upper-division undergraduate seminar taught at 
the University of Missouri in Columbia (MU) is described. The rationale for the 
curriculum design is that not only should scientific content and method be 
understood by students but they should also experience peer review as an integral 
and essential part of the process of science (ARISE 2, 2013; Bennett, Gadlin, & 
Levine-Finley, 2010; Coil, Wenderoth, Cunningham, & Dirks, 2010). The 
seminar course Scientific Writing in Chemistry integrates content, context, 
collaboration and communication in a unique fashion and meets the criteria for 
writing-intensive courses of the University of Missouri’s Campus Writing 
Program (2014). The curriculum was developed for a semester-long (14 weeks) 
course with three one-hour meetings per week and with limited enrolment. The 
curriculum was taught in the Spring semesters of 2010 - 2014 and results of 
evaluations for four years are presented. 

While the curriculum has been developed for students at an American research 
university, the framework of the assignment-based approach is entirely transfera¬ 
ble to other sciences and other educational levels and contexts. It is hoped that the 
present article will contribute to the wide dissemination of this curriculum. The 
curriculum described here is very much in the spirit of the Next Generation 
Science Standards (Achieve, 2013; Stage, Asturias, Cheuk, Daro, & Hampton, 
2013) and the report on integrated STEM education by the National Research 
Council (Honey, Pearson, & Schweingruber, 2014). 
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Framework of the curriculum and course organisation 

The types and sequence of the assignments and the modes of their assessment by 
various forms of peer review constitute the framework of the curriculum. These 
remain essentially the same from one implementation to the next (Table 1). 

Table 1 


Course design: Content, software, and resources (2014) 


Focus 

Week 

Task 

Points 

Peer 

Content 

Software and Online 





Review 


Resources 


1 




Reading Chemical 

Browser, Portals: 






Literature, Publication 

ACS, Wiley-VCH, 






Types 

RSC 

bo 

.R 

2 

A01 

20 

SR 1 

Mindmapping & 

Outlining, Text 

Word 

3 

A02 

20 

SR 1 

Schemes; Integration of 

ChemDraw, Word 

1 





Text & Art 



4 

A03 

20 

SR 1 

Tables, Statistics & 

Excel 

R 





Graphing 


to 

5 

A04 

20 

SR 1 

Simulation & Graphing 

Excel, Word, 

s 






ChemDraw, 

R 

6 

A05 

20 

SR 1 

Search, Citation & 

SciFinder, Word, 

| 





Bibliography 

ChemDraw 

> 

7 

A06 5 

20 


Oral Presentation 

Powerpoint 

s 

CO 

8 



MR 2 

Oral Presentation Week 


9 

A07 

20 

SR 1 

Structure and Modelling 

Chem3D, Jmol etc. 


10 

A08 

20 

SR 1 

Writing I. Materials, 

JOC Guidelines for 

bo 

.R 





Methods, Appendix 

Authors 

O 

•s -s 

11 

A09 


3FFF 3 

Writing II. Body & 

Authentic Examples 

O “S3 





Abstract, Cover Letter 

provided 


12 

A10 



Scientific Peer Review 

Ethics Guidelines: 

H ^ 
k) bo 

R 






ACS and NAS 

1 s 

13 

All 

40 

3FR 4 

Revising & Responding to 

Authentic Peer 

-R 





Peers, Graphical abstract 

Review Examples 

Tf 






provided 

A 3 
< <o 

14 




Scientific Conduct and 

PR-Cases & ORI- 





Misconduct 

Resources 


Notes to Table 1. 

1. SR refers to single & rubric-based peer review. 

2. MR refers to multiple & rubric-based peer review. 

3. 3FFF refers to three-fold free format peer review. 

4. 3FR refers to three-fold mbric-based peer review. 

5. In SP10, the oral presentation was “Project #1” and it became A06 in subsequent 
implementations. Hence, A06 - A10 in SP10 correspond to A07 - All in later years. 
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Each implementation of this framework presents a unique curriculum because all 
assignments are original with revised online resources and rubrics for assessment. 
In each semester, the assignments are connected to an overarching theme of the 
course: 

• SP10 (Spring 2010): Aspirin and other Painkillers 1 ; 

• SP11 (Spring 2011): Dyes, Indicators and Chemical Sensors 2 ; 

• SP12 (Spring 2012): Soaps, Detergents and Ambiphiles 3 ; 

• SP13 (Spring 2013): Solar Energy 4 ; 

• SP14 (Spring 2014): Nutraceuticals: Sources, Delivery and Functions 5 . 

All the assignments, associated data and sources, peer review devices and samples 
of completed assignments are available online and the URLs for the course web 
sites are provided as footnotes. The learning goals of the theme-based, research- 
oriented curriculum are well aligned with modern pedagogical principles. The 
framework of the curriculum emphasises crosscutting concepts (such as structure 
& function, pattern recognition, cause & effect), informs about scientific practices 
and provides instruction about all aspects of actual research (National Academy of 
Engineering, 2010). 

Writing a scientific research paper for a journal is very difficult and requires 
proficiency in many skills. Evidence suggests that it is most effective to move 
gradually from relatively simple tasks to more complex tasks in the writing 
process (Massengill, 2011; Topping, 2003). Because of this, more than half of the 
curriculum for this course is dedicated to Skill Development for Scientific Writing 
(A01 - A07, Table 1). After this preparation, the students engage in a Near- 
Authentic Exercise in Scientific Writing and Authoring (A08 - All, Table 1). 

In Assignments A01 - A04, students learn about outlining and organisation, about 
writing paragraphs, about the creation of schemes and figures, and about working 
with data (simple statistics and graphing). These assignments require the use of 
browser software, word-processing software, molecular structure drawing 
software, and spreadsheet software. Assignments A05 - A07 are more complex 
tasks with regard to content knowledge as well as process skills. Working on these 
assignments builds on the skills learned in A01 - A04 and also requires additional 
learning about literature and database access software, presentation software, and 
molecular display and modelling software. Moreover, the students need to raise 
their reading comprehension of primary literature, elevate their written 
communication skills, and they also need to leam the principles of, and 
demonstrate a proficiency in, persuasive oral presentations. 

The incremental increase in the complexity of the tasks is reflected in the gradual 
increase in the students’ autonomy regarding their selections of topic and sources. 

1 http://faculty.missouri.edu/~glaserr/RG_T_SP 10.html 

2 http://faculty.missouri.edu/~glaserr/RG_T_SP 11 .html 

3 http://faculty.missouri.edu/~glaserr/RG_T_SP 12.html 

4 http://faculty.missouri.edu/~glaserr/RG_T_SP 13.html 

5 http://faculty.missouri.edu/~glaserr/RG_T_SP14.html 
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In assignments A01 - A04, the students work on common topics and they select 
literature from a provided pool of sources about the theme area. In Assignments 
A05 - A07, the students work on different topics and they select one topic from a 
provided pool of pre-selected topics within the theme area. Finally, in 
Assignments A08 -All, the students work on different topics that they select 
freely from the primary literature covering the theme area and a pool of journals is 
provided to facilitate their access to the primary literature of theme area. 

Thus, for Assignments A01 - A04, most of the data and literature are provided and 
the focus is on “working with data and sources” on a common topic. Assignments 
A01 and A02 introduce the theme of the course. The number of sources supplied 
for AO 1 and A02 and the mode of their provision has evolved from providing a 
small number of articles with the assignment (SP10 - SP12) to providing a much 
larger collection of articles on the theme of the course in the section “Science 
Topic Resources” on the course web site (since SP13). This evolution was made 
possible by the realisation that students are quite able to deal with a larger pool of 
sources if they are taught strategies to browse them for leading ideas. 

In Assignments A03 and A04, students leam how to tabulate data, perform 
descriptive statistics, create histograms, and plot functions (A03) and to employ 
these skills for the reproduction of published spectra by sums of Gaussian 
functions (A04). All students work on the same data set for A03 to learn the basic 
skills and Assignment A04 is based on a recently published research article in the 
theme area of the course. With Assignments A05 and A06, the students take the 
step to “locating data and sources” about a topic of their choice selected from a 
list of possible topics within a defined theme. 

Assignment A07 serves as an introduction to structure visualisation and molecular 
modeling and this assignment is built around a protein that plays a key role in the 
theme area of the course: 

• SP10 (Spring 2010): COX-1 & COX-2 

• SP11 (Spring 2011): Green Fluorescent Protein 

• SP12 (Spring 2012): Photosystems I & II 

• SP13 (Spring 2013): Subtilisin 

• SP14 (Spring 2014): Superoxide Dismutase 

While the topic of Assignment A07 is defined and the same for all students, the 
students need to write their assignments based on one article they select from a list 
of many “hits,” and they must justify their choice to their peers. 

For the Near-Authentic Exercise in Scientific Writing and Authoring (A08 - A11), 
the students enjoy great autonomy and select their topic freely from the primary 
literature covering the theme area. To facilitate the students’ access to the primary 
literature, a list of specialised journals is included with A08 to provide some 
initial guidance. 
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Scheduling, collaboration and benefits of peer review 

The assignment-based curriculum requires students to work intensively on 
increasingly complex tasks during the semester. Several strategies have been 
employed to enable the students to live up to the high standards and to succeed in 
the course. Of course, the organisation must be explained with clearly stated 
deadlines so that the course remains on schedule. 

The schedule is organised to a common “rhythm” (Figure 1). Work on 
assignments begins in Week 2 of the course, a new assignment starts every week, 
and work on each assignment is completed within two weeks. The students learn 
about context and background information pertinent to the assignment of the week 
in a lecture on Monday. The assignment is handed out (in hard copy) on 
Wednesday together with the rubric (also in hard copy) that will be used for the 
assessment of the assignment. The purpose of the assignment, the requirements 
with regard to content and formatting, and scheduling issues are clarified in 
conversation. On Fridays, the students meet in the computer laboratory and start 
working on their assignment. Technical issues are addressed either through mini¬ 
lectures by the instructor and/or a group of peer teaching assistants. Because of 
this schedule, the students become quite confident about the assignment by the 
end of the week; they understand the assignment, are clear about the purpose and 
the requirements and have learned all the skills required to complete the 
assignment on their own. Remaining questions can be addressed in class on 
Monday of the following week and the assignments are due the next day. The 
assignments are distributed for peer review on Wednesday and returned on Friday. 


Week 

Day 

Instruction Mode 

A01 

A02 

A03 

2 

Mon 

Lecture 

Background 




Wed 

Conversation 

Assignment 




Fri 

Comp. Lab. 

Work Task 



3 

Mon 

Lecture 

Questions? 

Background 



Wed 

Conversation 

Peer Review 

Assignment 



Fri 

Comp. Lab. 

Return PR 

Work Task 


4 

Mon 

Lecture 


Questions? 

Background 


Wed 

Conversation 


Peer Review 

Assignment 


Fri 

Comp. Lab. 


Return PR 

Work Task 

5 

Mon 

Lecture 



Questions? 


Wed 

Conversation 



Peer Review 


Fri 

Comp. Lab. 



Return PR 


Mon 

Lecture 





Figure 1. Course design: Sample organisation of assignment schedule 
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Upper-division students might miss class meetings for legitimate reasons (e.g., 
illness, conference, interview, or athletics) and it can be difficult for students to 
negotiate the tight schedule of the curriculum throughout the semester. Therefore, 
it was decided that all activities in this course are to be performed by pairs of 
students. This stratagem has the immediate organisational benefit that every group 
can stay on schedule in spite of an occasional absence. Moreover, this stratagem 
also offers peer support (Carter, Cushing, & Kennedy, 2008) and several 
pedagogical advantages (Barkley, Cross, & Major, 2005; Hmelo-Silver, Chinn, 
Chan, & O’Donnell, 2013). The curriculum of this course is challenging and it is 
unlike anything the students have been confronted with in previous courses both 
in content and format. The novelty of the curriculum and the relevance of the 
themes certainly contribute to the students’ excitement about the course. 

However, searching for and reading primary literature and working on research- 
oriented assignments may lead to some apprehension. We have found that 
working in pairs greatly helps the students to manage the demands of the course 
and to alleviate any doubts they might have as to whether they can succeed. Each 
pair of students works collaboratively on all the assignments and they receive the 
same score for their joint work. Working together on the assignments and on the 
peer reviews harvests the benefits of peer-to-peer learning (Rheingold, 2014; 
Vazquez et ah, 2012) and especially promotes learning through collaborative 
argumentation (Chinn & Clark, 2013). 

Assessment of assignments by various forms of peer review 

Peer review has been criticised because it frequently fails to catch errors and 
fraud, may favour articles that maintain the status quo, may introduce bias and 
prejudice to the publication process, and may be used for “turf protection” (Park, 
Peacey, & Munafo, 2013; Watts, 1995; Ziman, 1968).Viewing peer review as 
merely a tool for quality control (Knoll, 1990; Rennie, 2003) might explain why 
peer review is largely absent from science education (Martyn, 2003). However, 
the objective of science is not just to acquire pieces of knowledge, rather its goal 
is a rational consensus over the widest possible field (Ziman, 1968) and the 
discourse leading to this goal therefore is the essence of science. As Ziman (1969, 
p. 324) explained, “It is not enough to observe, experiment, theorize, calculate and 
communicate; we must also argue, criticize, debate, and expound, summarize, and 
otherwise transform the information that we have obtained individually into 
reliable, well-established, public knowledge.” Further, Habermas (1991) stressed 
the significance of establishing public knowledge through public debate. If 
laypeople in a society do not accept the products or procedures of the systems’ 
sphere (science and technology), then the systems’ sphere loses its authority and 
its discoveries become meaningless in the context of the wider society. Thus, the 
greatest meaning of the process of discourse is to make the information corporate 
and communal knowledge through debate and persuasion. 

Most scientists have their first experience with peer review when they publish 
their first paper and receive peer review. For working scientists, however, both 
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receiving and providing peer reviews are regular activities (Nicol, Thomson, & 
Breslin, 2014). The curriculum design described here includes activities that 
exercise both directions of peer review in a gradually evolving fashion over the 
course of several months. We hold that a thorough development of peer review 
skills is intrinsically connected to the development of writing skills and argue that, 
in the absence of well-developed writing skills and some experience in scientific 
writing, one cannot judge the creative works of others “as a peer.” It takes much 
knowledge (Yankulov & Couto, 2012) and more than a briefing (Russell, 2013) to 
become proficient in scientific peer review. Peer review is much more than merely 
the ability to recognise substandard work and, therefore, peer reviewers should be 
able to appreciate the quality of original ideas. To build this capacity for 
constructive feedback on original works requires significant content expertise and 
writing practice. Meaningful curricula for science writing and peer review thus 
can be connected to laboratory courses (Walker & Sampson, 2013) or to lecture 
courses with substantial (Carson & Glaser, 2010; Hume et al., 2006) or intensive 
writing components. 

Several forms of peer review are employed in this course (see Table 1). The peer 
review tasks evolve from rubric-based peer assessments to free-format peer 
review. At the same time, the peer review tasks evolve from assessments of the 
writer’s technical and formal proficiencies and of the completeness of the 
assignment all the way to an assessment of the writer’s capacities for excellence 
in topic selection, for logical organisation and sequencing, for the logical 
construction of arguments and their clear presentation, and for sound judgements 
in the formulation of conclusions. As an example, the rubric employed for 
Assignment A05 in the spring semester of 2014 is shown in Figure 2. 

Scoring rubrics (ERIC, 2014; Panadero & Jonsson, 2013; Reddy & Andrade, 

2010) include several criteria (dimensions) for the assessment and a rating scale 
for each criterion (levels). The expected performance for each level is described 
(descriptors) in an assessment grid. We employ a 4-level rating scale comprising 
“Beginning” (1), “Developing” (2), “Accomplished” (3), and “Exemplary” (4). 
The number of criteria varies. For example, Assignment A01 has 10 criteria; 
Assignments A02, A03, A05, A08 have 7; Assignments A06 and Al 1 have 6; 
Assignment A07 has 5; Assignment A04 has 4 criteria as more space is allowed 
for commentary (A 10 is all free format). The types of criteria also gradually 
change from simple skill assessment early on to criteria for the assessment of 
higher-level issues. To define reasonable expectations and to set standards, 
samples of exemplary work by earlier cohorts of students (on assignments of the 
same types but on different topics) are posted on the course web site. All peer 
reviews are provided anonymously and the peer reviewers are not rewarded. 
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Peer Review A05 by 


(author’s names) 



Beginning 

1 

Developing 

2 

Accomplished 

3 

Exemplary 

4 

Score 

Identity, 
structure, 
uses, etc. 

Fewer than four 
parts answered. 

Five parts 
answered. IUPAC 
name not reported 
under problems 
with one Scheme. 

Five parts answered. 
Synonyms include 
IUPAC name. 

Schemes 1 & 2 
present. 

Six parts answered. 
Synonyms include 
IUPAC name. 
Schemes 1 & 2 
excellent. 

/4 

Experimenta 

1 section 

Lacking in scope, 
content and 
format. 

At least two 

analytical 

methods 

incomplete and/or 
incorrect data 
sets. 

Three sets of 
analytical data. 
Complete and correct 
data sets. Consistent 
formatting. 

Four sets of 
analytical data. 
Complete and 
correct data sets. 
Flawless formatting. 

/4 

Synthesis/Pr 

ocessing 

Incomplete/incorr 
ect structural 
drawings; unclear 
role of 
compounds 
drawn. 

Substrates and 
products drawn, 
but with some 

errors. 

Intermediates 
and/or reagents 
missing/incorrect. 

Substrates and 
products drawn 
correctly. Some 
intermediates and/or 
reagents 

missing/incorrect. 

Substrates, 
intermediates and 
products drawn 
correctly. Reagents 
clearly specified 

/4 

Art Citation 
& Caption 
Counts 0.5 

Schemes/Figures 
(S/F) are not cited 
in the text. No S/F 
captions in the 
text. 

Some S/F are 
cited in text. Every 
S/F has a caption. 
Caption formatting 
needs revision. 

Most S/F are cited in 
text. Every S/F has a 
caption. Captions are 
part of the text, 
formatted well. 

All S/F are cited in 
text. Every S/F has 
a caption. Captions 
are part of the text 
and formatting is 
flawless. 

12 

Ref. 

Citations 
Counts 0.5 

Some references 
are cited, some 
are not. Confused 
about concepts 
and/or 
conventions. 

References are 
cited. Placement 
in text and/or 
formatting needs 
revision. 

All references are 
cited using correct 
format. Placement in 
text is logical for most 
references. 

All references are 
cited using correct 
format. Placement 
in text is logical. 

12 

Bibliography 
Counts 0.5 

Lacking in 
substance, 
content and 
format. 

Appropriate 
references cited. 
Refs. Incomplete 
and/or poorly 
formatted. 

All but one of the 
references are 
complete. Formatting 
is not perfect but 
consistent. 

References are 
complete and 
formatting is 
flawless. 

12 

Sections 
Counts 0.5 

No idea about 
sections. 

Sections present. 
Problems with 
placement of 
references. 

Three sections. 
Problems with 
numbering of 
references 

Three sections, 
each on new page. 
References follow 
section, continuous 
numbering. 

12 

Constructive comments to guide the Authors’ Revision: 


Figure 2. The rubric employed for the assessment of Assignment A05 in the 
spring semester of 2014. 
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Rubric scoring is employed in the peer reviews of all assignments for Skill 
Development for Scientific Writing (A01 - A07) (see Table 1). In this phase, each 
assignment is assessed by single rubric-based peer review (SR) and the peer 
review is managed to ensure that each group reviews another group only once to 
minimise the consequence of overly positive or negative reviews. The oral 
presentations (A06) occur in three class meetings with 4-5 presentations per 
meeting. The presentations are assessed by multiple rubric-based peer review 
(MR) by the students in the audience (excluding all presenters and the co-chairs of 
the session) and the average score counts as the presenters’ A06 score. With the 
rubric-based peer reviews given and received in A01 - A07, the students are well 
prepared for the Near-Authentic Exercise in Scientific Writing and Authoring and 
its three-fold, journal-style scientific peer review. 

Scientific writing & peer review: Manuscript preparation, review and 
revision 

The overwhelming majority of scientific papers contain original data. Aspects 
related to the acquisition of the data are usually described in the Materials and 
Methods section and the data themselves are usually documented either in the 
paper or an Appendix or Supporting Information. An authentic exercise in 
scientific writing must be concerned with the rational analysis of original data 
within the existing context. However, there are obvious limitations to original data 
generation in writing classes and it is difficult for students to write a scientific 
paper without original data. 

To resolve this conundrum, we ask the students to identify a suitable topic in the 
theme area, to collect all relevant information about a recently described molecule 
or material (structure, preparation, chemical characterisation, performance) and to 
adopt these data as their own. Hence, the students browse recent primary literature 
in search of a subject of their choice. A list of professional journals in the theme 
area is, however, provided for initial guidance. Each group is required to provide 
all sources about their molecule or material as a bibliography on the last page of 
the appendix of their paper. 

With this premise, the sequence of Assignments A08 - All creates a near- 
authentic experience in scientific writing and its assessment by scientific peer 
review. As the students select their topics, they create rough outlines of their 
papers (mindmaps, keywords, sketches of basic concept), mindful of the most 
important questions: What is the primary function of their material? How does the 
material accomplish its primary function? How well does the material perform the 
intended function? Which chemical and/or physical properties are not desired in 
such a material and why? The manuscript preparation is perfonned in 
Assignments A08 and A09. 

Assignment A08 consists of: the selection of the subject, the collection of all the 
required information, and the documentation of experimental methods and results 
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in the materials and methods section and in the appendix of the projected paper 
using the author guidelines from the Journal of Organic Chemistry (JOC). The 
students should report details about the preparation of the new material and data 
documenting the spectroscopic characterisation as Supporting Information. In the 
body of the paper, the students must write a summary of the preparation and focus 
on the quantification of the performance of the material. Experimental details of 
measurements of the material’s primary function should be given in the Materials 
and Methods section and any data in support of the material’s performance must 
be reported in the body of the paper. Assignment A08 is evaluated by rubric-based 
peer review to ensure that the groups are on track before they complete their 
papers for Assignment A09. 

In Assignment A09 the sections, namely, Introduction, Results and Discussion, 
Conclusion, and Abstract are added to the product of A08. The students compare 
characteristic features of their material to two other prominent materials with the 
same function. With A09, the focus shifts from reporting and documenting to 
analysing, explaining, judging and concluding. In addition, with A09, the 
instruction on publication correspondence begins and the students need to write a 
cover letter to accompany the submission of the paper. 

Each group writes a paper on the general theme of the course. However, the 
papers vary greatly because of the students’ original selections of their specific 
topics. It is this commonality of the general theme together with the variety of the 
specific topics that ensures competent peer review. Hence, Assignments A10 and 
All truly provide instruction and practice in scientific peer review. Assignment 
A10 consists of the three-fold free-format scientific peer review (3FFF) of A09 
submissions following the peer review format and criteria of JOC. In Assignment 
A11, the students respond to the peer reviews received. They revise their 
manuscripts, write rebuttal letters (i.e., cover letters that describe and justify all 
changes made), and submit these items for a second round of three-fold rubric- 
based (3FR) peer review by the previous three referees. This review of the revised 
papers is a rubric-based peer review and the average score of three reviews 
becomes the All score. 

Grading: Encourage high quality original submissions 

Initially (Spring semesters of 2010 - 2012, SP10-SP12), student course grades 
were based on their completion of all assignments in an “acceptable manner” (> 
15/20 points by peer review after revision). With growing confidence in the 
quality and fairness of the peer review process, the grading procedure was 
modified to increasingly reflect the peer review scores. Here we describe the 
grading scheme implemented since the Spring Semester 2013. 

The peer review of an assignment results in a peer review score (PRS) up to a 
maximum of 20 points and the authors receive the PRS together with the rubric(s) 
completed by their peer reviewers. Various modes of revision are required 
depending on the peer review score of an original submission (PRSO). High 
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quality original submissions are valued and rewarded during the Skill 
Development for Scientific Writing phase of the course. No revision is needed for 
a submission with PRSO >19 (“Accepted as is”)- Usually scores falls in the range 
19 > PRSO >15 (“Minor revision required”) and the students are asked to: 

1. to read the peer review comments carefully 

2. to revise their assignment considering the reviewer comments, and 

3. to submit the revision in electronic form to the instructor with changes 
made with “tracking on.” 

The assignment is completed once the instructor accepts a satisfactory revision. 
Peer review scores PRSO <15 (“Major revision required”) are rarely given and, if 
they are, the students are asked to perform steps (1) - (3) and to complete an 
additional task, namely, to describe the changes made and explanations as to how 
the changes address the comments by the peer reviewers. The revision is accepted 
once it scores above 15 points. 

In the Near-Authentic Exercise in Scientific Writing and Authoring phase of the 
course, again high quality original submissions of Assignment A08 are rewarded 
because all of the skills required for the execution of A08 have been acquired and 
practised in A01 - A07. However, the complete original paper is scored only after 
scientific peer review and revision, i.e. A11 is scored while A09 is not. 

Course grades are based on the student’s average of PRSO scores (<PRSO>) as 
follows: “A” if <PRSO> > 19, “A-” if <PRSO> > 18, “B+” if <PRSO> > 17, and 
so on. In addition, students can improve their grades by: exemplary work (posted 
as sample, seven opportunities); an outstanding oral presentation (top three 
presentations); and/or an outstanding final paper (top three papers). Every 
instance of a special recognition improves the student’s grade by one “notch.” For 
example, a student with an average original peer review score of 17.5 and one 
submission selected for posting as a sample will receive an “A-”. 

The instructor shapes the outcome of the peer review by careful design of the 
assignments and the associated rubrics, by the provision of guidance, advice and 
feedback as the students work on their assignments, by discussion of posted 
exemplary samples from previous courses, and by the selection and recognition of 
exemplary submissions from the current course. However, to teach scientific peer 
review requires the instructor to create an atmosphere in which the students leam 
to trust the outcomes of peer review and, hence, it is important that the 
involvement of the instructor is minimised once the peer review process of the 
submissions proceeds. The instructor should only offer to mediate any disputes 
that might arise in the peer review. 

Results of evaluations 

Informal evaluations 

Direct contact with the students in class and especially in the computer laboratory 
and the conversations with the peer teaching assistants in weekly meetings with 
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the instructor provide continuous sources of formative feedback about all aspects 
of the course design and its implementation. Any concerns are identified and 
addressed in a timely manner and before significant problems can develop. 


Toward the end of the course, the students are asked to list the ten most pertinent 
keywords they associate with the course. The frequency analysis of the collected 
keywords provides a “wordle” (Feinberg, 2013) as exemplified in Figure 3. 
“Writing,” “Research,” and “Chemistry” are usually the top three associations 
although their relative frequencies vary from semester to semester. The fourth 
most frequently occurring word usually has to do with the theme of the course 
(i.e., “Solar” in SP13). 


Ti02 Spectra 



Scientific Pp C 

^nppr Ul^llO 

, |JCGI 


■UBK 

solar 

chemistry Formatting 

Energy 


ruthenium 

Graetzel 

Dye Smut i/ed Piper 


llClIb Dye's Lllclfey | “ 

Research 


DSSC 

Dye 

Po»w J elearode 


Absorption 

"■P^ChmOrwr 

Review 


Future . \ JST. Z907 SciFinder 
research I tJ VIUW Putilish JQQ 

“^writing ■ 



Figure 3. Wordle of the top-10 associations by the SP13 students at the end of 
the seminar “Scientific Writing in Chemistry” with the theme “Solar Energy.” 
Font size reflects the frequency of the word. 

This outcome of the free association exercise is nicely aligned with the intended 
learning goals. The course educates in scientific writing in chemistry and is 
research intensive. The themes of the courses facilitate these educational goals, 
and are selected to be sufficiently interesting and appealing to a general audience 
so that overly specialised chemistry content issues do not dominate the discourse. 
It also is noteworthy that some words occur only with low frequencies, and words 
associated with peer review fall into this category. Peer review is not a source of 
apprehension and, instead, the students accept and respect the peer review 
process. 

Formal evaluations 

The student’s teaching evaluation involves a three-part questionnaire. Through 
legislation, Missouri (Spieler, 2007) requires students to rate their teachers on 
three “Consumer Information” criteria (Table 2) using a 4-level Likert scale 
(“strongly agree”, “agree”, “disagree”, “strongly disagree”) with 4 being the 
highest score. Students also may select “no opinion,” The criteria are: 


2014 Vol. 7 No. 2 


97 


Special Issue: Science Education 








Journal of Learning Design 


Glaser 


1. The course content, including the lectures, syllabus, grading standards, and 
student responsibilities, was presented clearly. 

2. The instructor was interested in student learning. 

3. Considering both the possibilities and limitations of the subject matter and 
the course (including class size and facilities), the instructor taught 
effectively. 


Table 2 


Student Teaching Evaluations, Spring Semesters 2010 to 2013 (SP10-SP13) (on 4 
point scale) 


Criteria of Evaluation 

SP10 1 

SP11 2 

SP12 3 

SP13 4 

Consumer Information, SB 389, #1 

3.56 

3.67 

3.72 

3.79 

Consumer Information, SB 389, #2 

3.88 

3.86 

3.79 

3.96 

Consumer Information, SB 389, #3 

3.56 

3.38 

3.66 

3.58 


Notes to Table 2. 


1. Spring 2010: http://faculty.missouri.edu/~glaserr/3700sl0/3700sl0_evals.html 

2. Spring 2011: http://faculty.missouri.edu/~glaserr/3700sl l/3700sll_evals.html 

3. Spring 2012: http://faculty.missouri.edu/~glaserr/3700sl2/3700sl2_evals.html 

4. Spring 2013: http://faculty.missouri.edu/~glaserr/3700sl3/3700sl3_evals.html 

In addition, all students taking any course offered by MU’s Department of 
Chemistry are asked to rate the teacher on the seven criteria listed in Table 3 using 
a 5-level Likert scale (4 - 0, 4 is high). The overall rating is detennined as the 
average of the numerical scores of the seven department questions and this 
aggregate score is commonly used to measure teaching perfonnance. 

Table 3 


Student Teaching Evaluations, Spring Semesters 2010 to 2013 



Criteria of Evaluation 

SP10 1 

SP11 2 

SP12 3 

SP13 4 

1 

Organisation and preparation of lectures and discussions 3.73 

3.61 

3.90 

3.84 

2 

Instructor’s enthusiasm for the subject matter 

3.96 

3.96 

4.00 

3.96 

3 

Helpfulness in answering questions and clarifying points 3.50 

3.35 

3.76 

3.84 

4 

Ability to lecture in a manner which is easily followed 

3.46 

3.52 

3.76 

3.72 

5 

Ability to stimulate interest in the subject 

3.07 

3.22 

2.97 

3.72 

6 

Overall rating of the instructor 

3.69 

3.57 

3.66 

3.88 

7 

Your rating of how much you have learned 

3.34 

3.26 

3.34 

3.66 

8 

Overall rating, Average of Qs 1 - 7 

3.53 

3.50 

3.63 

3.80 


Notes to Table 3. 


1. Spring 2010: http://faculty.missouri.edu/~glaserr/3700sl0/3700sl0_evals.html 

2. Spring 2011: http://faculty.missouri.edu/~glaserr/3700sl l/3700sll_evals.html 

3. Spring 2012: http://faculty.missouri.edu/~glaserr/3700sl2/3700sl2_evals.html 

4. Spring 2013: http://faculty.missouri.edu/~glaserr/3700sl3/3700sl3_evals.html 
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In the third part of the questionnaire, the students are asked to respond to the 
following six questions: 

1. List the strong and weak features of the lecturer and include suggestions 
for improvement. 

2. Compare the lecturer to others you have had (especially with those in 
science courses at this level). 

3. List the strong and weak features of the overall course (not the lecturer) 
and include suggestions on how its quality might be improved. 

4. Compare the course with the others you have taken. 

5. Your overall rating of the course (circle letter grade): A B C D E. 

6. My approximate GPA prior to the current semester was_. 

The students fdled out the questionnaire toward the end of the semester and in the 
absence of the teacher. The teacher allocated 10-15 minutes of class time for the 
students to fill out the questionnaires. Table 4 provides data on student numbers, 
completion rates for the evaluations and retention for the course. 

Table 4 


Student Teaching Evaluations, Spring Semesters 2010 to 2013 


Criteria of Evaluation 

SP10 1 

SP11 2 

SP12 3 

SP13 4 

Enrolment 

32 

25 

36 

32 

Evaluations returned 

23 

23 

29 

25 

Per cent of students returning evaluations 

72% 

92% 

81% 

78% 

Student retention 

100% 

100% 

100% 

100% 


The results of the teaching evaluations (average scores on Likert scale items) and 
transcripts of verbatim comments to the above questions became available to the 
teacher after all grades were filed. The author has made it a practice to post all of 
these data (average scores and complete student comments) online and freely 
accessible. The URLs are provided as notes to Tables 2 and 3 and selected student 
comments are listed in Tables 5 and 6. 

The numerical results of evaluations show that the curriculum has enjoyed a high 
level of acceptance by the students every semester (Tables 2 - 4). The increase of 
the overall rating may reflect the gradual refinements in the selection and delivery 
of the background material as well as the availability of a growing body of 
exemplary samples from earlier semesters. 

Most importantly, the student commentaries (Tables 5 and 6) show that the 
curriculum is not only accepted but is welcomed and desired. In particular, the 
design of the peer review system has been succeeding very well (Table 6) with no 
more than 3 requests for the instructor to mediate peer review disputes in any 
given semester. 
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Table 5 

SP13 Students on Curriculum 

Student Student Comments 

SOI: It’s a great course that offers loads of research experience. 

S04: It was unique, making it interesting. 

S06: Different but relatively effective. 

S08: One of the most interesting and challenging courses I have taken. I wish more 

classes were like this one. 

S09: Very interesting topics but lots of work! 

SI7: Dr. Glaser chose a relevant and up and coming subject to study over the 

semester. Most professors teach from the book and don’t relate it to what is 
relevant and important in today’s society. 

SI8: Very good, always interesting class meetings. 

S19: Different but really effective. 

S21: Learned how to research a lot better. I learned a lot on my own, not in class, 

but that may have been the goal of the class. 

S22: I learned a lot. 


Table 6 

SP 13 Students on Peer Review 

Student Student Comments 

S16: The assignments were reasonable but I wasn’t [a] fan of the peer reviewing. 

S18: Peer reviews [comments by fellow students, i.e., in A11] not always explained 

in a way easy to follow. 

S23: Peer review grading seems to be an issue, as some students choose to be more 

rigorous than others. However, Dr. Glaser has a system for addressing issues. 

S23: I think a group exercise in conducting peer-reviews would help. 


Conclusion 

We developed, implemented and assessed an assignment-based curriculum for a 
writing-intensive seminar called Scientific Writing in Chemistry to instruct 
students about all aspects of the scientific publication process and peer review. 
The course organisation, the scheduling of the assignment activities and the 
stratagem of requiring peer collaboration create an educational environment that 
harnesses the benefits of peer support and enables the students to work on 
research-oriented tasks. The learning goals of this curriculum are compelling and 
self-evident. We believe that the high level of acceptance reflects the clarity of the 
purpose of the curriculum. 

The assignments start with simple tasks and evolve to more technically and 
intellectually challenging tasks and the principle of scaffolding is applied in 
various dimensions. The complexity of the writing tasks increases with regard to 
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scope and integration of text and art. The other dimensions of scaffolded 
evolution include the levels of difficulty of the science content, the levels of 
working with primary sources, and the level of peer review proficiency. 

The curriculum consists of a framework of assignments and this framework is 
defined by the types and sequence of the assignments and the modes of their 
assessment by peer review (Table 1). Every iteration of this framework is built 
around an overarching theme and, hence, every course presents a unique 
curriculum of original and connected assignments. The selection of a relevant and 
timely theme not only stimulates the students’ interests, curiosity and motivation, 
but this novelty also ensures fresh and engaged instruction in every 
implementation. This curriculum is entirely transferable to other disciplines and 
we encourage the readers to implement similar courses on Scientific Writing in 
“your-discipline-here. ” 
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